IMAGE COLLECTION APPARATUS AND METHOD 



FIELD OF THE INVENTION 

The present invention relates to an image collection 
apparatus and method to be used for monitoring the vicinity of 
a place such as a parking lot, a bank, a convenience store, a 
road side or the like where an image acquisition apparatus such 
as a video camera is installed or for delivering image 
information by means of the image acquisition apparatus. More 
specifically, this invention relates to an image collection 
apparatus capable of rapidly retrieving and displaying a 
desirable one of the images to quickly investigate the cause 
of an accident or a crime or of rapidly analyzing the state of 
the vicinity of the installation place based on the image. 

gACKGRQ U N D QF THE INVENTION 

In recent years, the image collection apparatus for 
c*o-riecting and monitoring an image of an installation place 
which is picked up by the installed video camera or the like 
has been increasingly required according to the necessity. As 
a conventional prior art related to such an image collection 
apparatus, for example, Japanese Patent Application Laid-Open 
No. 10-240774 discloses a data collection method and monitoring 
apparatus. In this apparatus, images of an event are stored 
just before and after the time at which the event is generated. 



Thus, a lot of events can be stored even if the storage capacity 
is small. The conventional image collection apparatus will be 
explained below with reference to the this apparatus. 

Fig. 28 is a block diagram showing the entire structure 
of the conventional image collection apparatus. A plurality 
of local devices 101, a plurality of center devices 102, and 
a data management device 103 are connected to each other through 
a network 105 and can mutually transmit and receive information . 
The local devices 101 are placed at places that are to be 
monitored. The center devices 102 are placed at a place where 
a supervisor (a person who is authorized to monitor ) is present. 
A display device 106 is provided to each center device 102, which 
can display the image information or reproduce voice 
information for the supervisor. The local device 101 has a 
video camera, which acquires image information and voice 
information. The local device 101 stores the acquired 
information in a storage device 104 . The data management device 
103 intensively manages stream index information about the 
location of information (stream data) recorded in a media on 
the network 105, for example, the storage device 104. The 
center device 102 retrieves the stream data index information 
based on certain retrieval conditions, obtains the result of 
retrieval for the location of the stream data to know the 
location of the stream data satisfying the retrieval conditions 
and collects stream data satisfying the retrieval conditions 



from the media. 

A data input unit 107 digitizes the acquired image 
information and voice information. The image information is 
coded into MJPEG (Motion Joint Photographic Coding Experts 
Group) or MPEG (Motion Picture Experts Group) and the voice 
information is coded into PCM (Pulse Code Modulation) and ADPCM 
(Adaptive Differential Pulse Code Modulation) . An input data 
storage and read unit 108 stores the coded data as timely 
continuous stream data in the storage device 104 . In addition, 
the input data storage and read unit 108 transmits the stored 
stream data through a communication unit 109 and the network 
105 according to a request from the center device 102 through 
the network 105. 

In this case, the input data storage and read unit 108 
generates stream data index information causing the input image 
information and voice information to correspond to the 
generated event, and stores the stream data index information 
in the storage device 104 and transmits the same to the data 
management device 103. In the case in which desirable image 
information or the like is fetched as described above, the 
stream data index information stored in the data management 
device 103 is used. 

The input data storage and read unit 108 stores, as event 
information, the stream data on the image information and the 
voice information for only a predetermined time before and after 



the generation of the event. For instance. Fig. 29 shows an 
example of the stream data index information. As shown in the 
column of recording time duration in the stream data index 
information Dl, image of the event 300 seconds before the 
generation of the event and 300 seconds after the generation 
of the event is stored. The stream data index information Dl 
includes event ID, type of the event, information about coding, 
information about the input device and the local device which 
picked the image, and the like. 

Fig. 30 is a diagram showing the recording state of an 
event image corresponding to the stream data retrieval 
information Dl in Fig. 29. Three local devices Loc03, Loc08 
and LoclO independently pick up images and record, as event 
images 111a to 111c, 300 seconds before and after the time at 
which an event is generated on the time base of each of all picked 
up images 112a to 112c. More specifically, the local devices 
Loc03 and Loc08 store the event images 111a and 111b at the same 
time for an event having an event ID of Ev003. For an event 
having an event ID of Ev00 6 generated later, the local device 
LoclO stores the event image 111c. Thus, only images picked 
up before and after the time of the event generation are stored. 
Consequently, even if the storage device 104 has a limited 
storage capacity, a lot of information can be stored- An event 
here means an event occurring instantly and does not have a time 
width. The same definition holds in the following description. 



However, the conventional image collection apparatus 
stores the image picked up for a predetermined time mainly 
including the time at which an event is generated- Thus, 
individual event images are stored independently of other event 
images. If a group of related events is to be retrieved, a lot 
of time is required for the retrieval operation. Even if it 
is considered that the retrieval operation does not require long 
time, it is difficult to grasp the relationship of the events 
in the group on a time basis. Thus, there is a disadvantage 
that it requires time and labor to carry out the monitoring or 
analysis . 

These problems are explained in detail with respect to 
Fig. 31 and Fig. 32. Fig. 31 shows the relationship between 
an event image including a series of related event images and 
an event. In the whole image 112 in Fig. 31, five events of 
entry into and leaving from a room, that is events 121 to 125 
(event ID: 1 to 5) , are generated and event images 113 to 117 
are recorded corresponding to the events 121 to 125. 
Accordingly, when events are retrieved with the type of the 
event as "entry", for example, the event images 121, 122 and 
124 can be fetched and referred. 

Consider, as shown in Fig. 32, that the events 121 and 
123 correspond to entry and leaving of the same employee 
(employee Nal22) . In this case, it is impossible to refer to 
a macro event image 141 corresponding to a bigger event (which 



will be hereinafter referred to as macro event) which is an event 
having a time width the vicinity of a time including the time 
of the generation of the event 121 at which this employee enters 
the room to the vicinity of a time including the time of the 
generation of the event 122 at which this employee leaves the 
room. Similarly, consider that the events 122 and 125 
correspond to entry and leaving of different employee (employee 
No.l55) . In this case, it is impossible to refer to a macro event 
image 142 corresponding to a macro event having a time width 
from the vicinity of a time including the time of the generation 
of the event 122 at which this employee enters the room to the 
vicinity of a time including the time of the generation of the 
event 125 at which this employee leaves the room. 

SUMMARY OF THE INVENTION 

It is an object of this invention to obtain an image 
collection apparatus and method capable of rapidly retrieving 
and displaying a desirable image without a labor by 
automatically collecting an image within a time at which a 
related event group is generated and of monitoring and analyzing 
the state of an image pick-up site based on the collected image 
rapidly and easily. 

In order to achieve the above-mentioned object, according 
to one aspect of the present invention, an image acquisition 
unit acquires an image, a event detection unit detects an event, 



and a event data creation unit creates event data indicative 
of at least a generation time and contents of the event detected 
by the event detection unit. A macro event definition unit 
which defines a macro event including a related event group and 
5 having a time width by using the generation time and contents 
of the event, a macro event data creation unit creates macro 
event data by structurizing a plurality of event data based on 
definition contents of the macro event, and an image recording 
i;3 unit records an image of a macro event corresponding to the macro 

in 10 event data. A retrieval unit retrieves a desirable macro event 
•;2 based on the macro event data, and a display unit acquires at 

s 

least an image of the macro event retrieved by the retrieval 

m unit from the image recording unit and displays the image . 

i ft 

i^i] The apparatus further comprises a temporary storage unit 

15 which temporarily stores the image acquired by the image 
acquisition unit, the image recording unit extracting and 
recording an image related to an event detected by the event 
detection unit from the image temporarily stored by the 
temporary storage unit. 
20 Further, a macro event definition unit carries out 

definition including repetition of a predetermined event in a 
series of events constituting the macro event and the macro 
event data creation unit creates macro event data in accordance 
with the definition contents. 
25 The apparatus further comprises an image summary unit 



which selects an image in a predetermined portion of the image 
of the macro event and summarizes the image of the macro event, 
the image recording unit recording the image of the macro event 
summarized by the image summary unit. 

The apparatus further comprises an image summary unit 
which selects an image in a predetermined portion of the image 
of the macro event, periodically acquires a static image from 
a non-selected image of the macro event and connects the 
selected image with the static image on a time basis, thereby 
summarizing the image of the macro event, the image recording 
unit recording the image of the macro event summarized by the 
image summary unit. 

Further, the image summary unit selects an image in a 
predetermined portion of the image of the macro event, 
periodically acquires a static image from a non-selected image 
of the macro event and connects the selected image with the 
static image on a time basis, thereby summarizing the image of 
the macro event, the image recording unit recording the image 
of the macro event summarized by the image summary unit, and 
particularly, intermittently interpolating the images 
selected by the static image. 

Further, the measurement unit measures data related to 
the image, and the measured data storage unit stores the 
measured data acquired by the measurement unit in relation to 
the macro event, the display unit displaying the image of the 
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macro event and displaying the measured data related to the 
macro event, thereby supporting to understand the contents of 
the image of the macro event. 

The apparatus further comprises a plurality of image 
acquisition devices, the image recording unit selecting one of 
the image acquisition units corresponding to the image of the 
macro event from a plurality of images input from the image 
acquisition unit and recording, as the image of the macro event, 
the image input from the selected image acquisition unit. 

The apparatus further comprises an analysis unit, which 
carries out a predetermined analysis processing based on at 
least the image of the macro event. 

According to another aspect of the present invention, an 
image is acquired and an event is detected at an acquiring step, 
event data indicative of at least a generation time and contents 
of the event detected at the acquiring step are created at a 
event data creating step, macro event data are created by 
structurizing a plurality of event data generated by the event 
data generation unit based on macro event definition contents 
defining a macro event including a related event group and 
having a time width at a macro event data creating step, an image 
of a macro event corresponding to the macro event data is 
recorded at a image recording step, a desirable macro event is 
retrieved based on the macro event data at a retrieving step, 
and at least the image of the macro event retrieved at the 



retrieving step is acquired and displayed at a displaying step. 

The method further comprises a temporary storing step of 
temporarily storing the image acquired at the acquiring step, 
the image recording step extracting and recording an image 
related to an event detected at the acquiring step from the image 
temporarily stored at the temporary storing step. 

Further, the macro event creating step creates macro 
event data based on the macro event definition contents defined 
including repetition of a predetermined event in a series of 
events constituting the macro event. 

The method further comprises an image summarizing step 
of selecting an image in a predetermined portion of the image 
of the macro event and summarizing the image of the macro event, 
the image recording step recording the image of the macro event 
summarized at the image summarizing step. 

The method further comprises an image summarizing step 
of selecting an image in a predetermined portion of the image 
of the macro event, periodically acquiring a static image from 
a non-selected image of the macro event and connecting the 
selected image with the static image on a time basis, thereby 
summarizing the image of the macro event, the image recording 
step recording the image of the macro event summarized at the 
image siammarizing step. 

The method further comprises a measuring step of 
measuring data related to the image, and a measured data storing 
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step of storing the measured data acquired at the measuring step 
in relation to the macro event, the displaying step displaying 
the image of the macro event and displaying the measured data 
related to the macro event. 

Further, the image recording step selects one of the 
images corresponding to the image of the macro event from a 
plurality of input images and records the selected image as the 
image of the macro event. 

Other objects and features of this invention will become 
apparent from the following description with reference to the 
accompanying drawings . 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing a complete structure 
of an image collection apparatus according to a first embodiment 
of the present invention; 

Fig. 2 concretely shows the arrangement of the image 
collection apparatus according to the first embodiment 

Fig. 3 shows the structure and specific example of event 

data; 

Fig. 4 explains the relationship between the image 
pick-up region of an image acquisition section and a region, 
which detects an event; 

Fig. 5 explains the relationship between the whole image, 
an event and an event image; 



Fig. 6 is a diagram showing the specific example of macro 
event definition contents according to the first embodiment; 

Fig. 7 is a flowchart showing the control procedure for 
an event processing and a micro event processing which are 
carried out by a control section of the image collection 
apparatus according to the first embodiment; 

Fig. 8 is a diagram showing the specific example of event 
data including an event to be used for the structure of the macro 
event; 

Fig, 9 is a diagram showing the specific example of macro 
event data; 

Fig. 10 explains the relationship between an event, a 
whole image, an event image and a macro event image; 

Fig. 11 is a diagram showing the relationship between an 
event and a macro event, which is obtained when a plurality of 
image acquisition sections, are provided; 

Fig. 12 is a diagram showing an example of a display screen 
for displaying information about the event and the macro event; 

Fig. 13 is a diagram showing an example of the input of 
retrieval information for retrieving event data; 

Fig. 14 is a diagram showing an example of a display screen 
indicative of a state in which the event data is selected and 
an event image is displayed; 

Fig. 15 is a block diagram showing a complete structure 
of an image collection apparatus according to a second 
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embodiment ; 

Fig. 16 is a diagram showing the specific example of macro 

event definition contents according to the second embodiments- 
Fig . 17 is a flowchart showing the control procedure for 

an event processing and a macro event processing which are 

carried out by a control section of the image collection 

apparatus according to the second embodiments- 
Fig. 18 is a diagram showing the specific example of macro 

event data according to the second embodiment; 

Fig. 19 is a block diagram showing a complete structure 

of an image collection apparatus according to a third 

embodiments- 
Fig. 20 is a diagram showing a predetermined time at which 

recording is permitted in the case in which a macro event image 

is to be recorded for a long period of time; 

Fig. 21 is a diagram showing an example of the summary 

of a macro event image according to the third embodiment; 

Fig. 22 is a diagram showing an example of the summary 

of the macro event image according to the third embodiment; 

Fig. 23 is a block diagram showing a complete structure 

of an image collection apparatus according to a fourth 

embodiment; 

Fig. 2 4 is a diagram showing an example of measurement 
data according to the fourth embodiment; 

Fig. 25 is a diagram showing an example of the display 



of the measurement data to be displayed according to the fourth 
embodiment; 

Fig. 2 6 is a block diagram showing a complete structure 
of an image collection apparatus according to a fifth 
embodiment; 

Fig. 27 concretely shows the arrangement of the image 
collection apparatus according to the fifth embodiment; 

Fig. 28 is a block diagram showing a complete structure 
of a conventional image collection apparatus; 

Fig. 29 is a diagram showing the specific example of 
stream data index information created by the conventional image 
collection apparatus ; 

Fig. 30 is a diagram showing the relationship between a 
whole image picked up by a plurality of local devices in the 
conventional image collection apparatus and an event image; 

Fig. 31 is a diagram showing the relationship between 
related events and the event image; and 

Fig. 32 is a diagram showing a relationship between 
related events, event image and a macro event image. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Preferred embodiments of an image collection apparatus 
and an image collection . method according to the present 
invention will be described below in detail with reference to 
the accompanying drawings . 



First Embodiment 

A first embodiment of the present invention will be 
described below. Fig. 1 is a block diagram showing the whole 
structure of an image collection apparatus according to the 
5 first embodiment of the present invention. In Fig. 1, an image 
acquisition section 1 is implemented by a video camera or the 
like and serves to pick up the image of a predetermined subject 
and to transmit the picked up image to a control section C. An 
O event detection section 2 serves to detect an event generated 

i-^^ 10 in relation to the image acquired by the image acquisition 
'-3 section 1 and to transmit information about the detected event 

r"^ to an event data creation section 3 through the control section 

h ^- 

The event data creation section 3 serves to create event 

lit 

15 data based on information about the event detected by the event 
detection section 2, that is, information about an event such 
as a time at which the event is generated, the type of the event 
and the like. A macro event creation section 4 serves to create 
macro event data which is obtained by structuring a plurality 

20 of event data created by the event data creation section 3. A 
macro event definition section 5 has macro event definition 
contents 5a which define a structured model to be referred when 
the macro event creation section 4 is to create the macro event 
data . 

25 A management data section 7 serves to hold and manage 
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event data 7a created by the event data creation section 3 and 
macro event data 7b created by the macro event creation section 
4. An image recording section 8 serves to record an event image 
corresponding to the event data created by the event data 
creation section 3 and to record a macro event image 
corresponding to the macro event data created by the macro event 
creation section 4. The image recording section 8 has a 
temporary storage section 8a. The temporary storage section 
8a serves to continuously and temporarily record the image 
acquired by the image acquisition section 1, and to erase the 
temporarily recorded image after a predetermined time. 

A retrieval section 6 serves to specify a retrieval object 
based on the event data 7a or the macro event data 7b which is 
managed by the management data section 7 and to fetch, from the 
image recording section 8, an image to be the specified 
retrieval object. An operation section 9 is implemented by a 
keyboard or a pointing device which inputs various operations, 
and serves to input the macro event definition contents 5a, 
retrieval conditions and the like. A display section 10 
implements a graphical user interface which is integral with 
the operation section 9 and serves to display and output the 
contents retrieved by the retrieval section 6, particularly, 
the retrieved image. The control section C serves to connect 
the above-mentioned sections and to control the whole 
apparatus . 



Although only one image acquisition section 1 and only 
one event detection section 2 are shown in Fig. 1, a plurality 
of image acquisition sections 1 and a plurality of event 
detection sections 2 may be provided in this image collection 
5 apparatus. The number of the acquisition sections 1 and the 
event detection sections 2 is optional. That is, one 
acquisition section may be provided with respect to plural event 
detection sections, or plural acquisition sections may be 
a provided with respect to one event detection section, or plural 

m 10 acquisition sections may be provided with respect to plural 

"''-■4 

event detection sections. 

Fig. 2 concretely shows the arrangement which implements 
the image collection apparatus according to the first 
:=y embodiment. Here, the image collection apparatus shown m Fig. 

15 1 is applied as an apparatus for monitoring (monitor apparatus) 
a parking lot . A camera 21 corresponds to the image acquisition 
section 1 and serves to pick up the contents of an image in a 
predetermined region in a parking lot 20, for example, the image 
of the parking of a car 23 for the predetermined region and to 

20 transmit the image to a partial image collection apparatus 22 
through a network N. The partial image collection apparatus 
22 which comprises and houses all the components of the image 
collection apparatus shown in Fig. 1 except the image 
acquisition section 1. The partial image collection apparatus 

25 22 is implemented by a computer or the like. The image picked 
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up by the camera 21 may be a dynamic image and/or a static image. 
The camera 21 may collect voice information along with the image 
information. Although the event detection section 2 is 
included in the image collection apparatus body 22 to detect 
an event based on a change in the contents of the image picked 
up by the camera 21 and the like, a sensor may be provided in 
the parking lot 20 to detect an event. 

The operation of each section in the image collection 
apparatus shown in Fig. 1 will be described below considering 
as an example a case in which the parking lot 20 shown in Fig. 
2 is monitored. Fig. 3 shows an example of the event data 7a 
created by the event data creation section 3. As shown in this 
figure, the event data 7a includes data corresponding to each 
of items such as event ID 31a, event type 31b, event location 
31c, event object 31d, event time 31e, camera ID 31f, image 
recording start time 31g, image recording end time 31h, static 
image 31i and dynamic image 31 j . 

The event ID 31a is a serial number provided to a series 
of events detected by the event detection section 2. The event 
ID shown in Fig. 3 is 1, and it is apparent that the event is 
the first one that is detected by the event detection section 
2. The event type 31b indicates the type of the event. The 
event type 31b is represented by numerals. 

Since this example explains monitoring operation of the 
parking lot, following conditions are assumed for convenience. 

18 



Let event type "'1" denotes "car entry (into the parking lot)". 
That is, this event type means that a car enters into the image 
(precisely, into an image pick-up region) . Let event type ''2" 
denotes ''car parking (into the parking lot)". That is, this 
5 event type means that the car stands still in the image. Let 
event type "3" denote ''car departure (from the parking lot)" . 
That is, this event type means that that was standing still in 
the image starts to move . Let event type "4" denote "car leaving 
□ (from the parking lot)". That is, this event type means that 

M 10 the car which was in the image disappears from the image. 
=3 Finally, let event type "5" denote "person entry". That is, 

:y 
n 

this event type means that a person enters the image . The event 
type in Fig. 3 is "2". This means that the event corresponding 
^ to the event data 7a is "car parking". The event type 31b is 

:=a 

:=b 15 determined by the event detection section 2 by detecting the 

:3 

change in the data for the pixels in the image. 

The event location 31c indicates the place where the event 
takes place- More specifically, the event location 31c is a 
parking region in the parking lot and corresponds to the number 

20 of the parking region. Fig. 4 is an example of an image of a 
image pick-up region 32 picked by the image acquisition section 
1 . As shown in this figure, the image pick-up region 32 includes 
regions 33a to 33c which are parking regions for three cars in 
a parking lot. Assume that the event location corresponding 

25 to the regions 33a, 33b, and 33c are "1", "2", and "3", 

19 



respectively. The event location in Fig. 3 is ''3". This means 
that, the event ""'car parking'' is generated in the region 33c. 

The event detection section 2 determines the event 
location, in the same manner when detecting the event type, 
5 depending on whether the data for the pixels in any of the regions 
33a to 33c in the image (image pick-up region) has changed. The 
event detection section 2 writes a data having meaning as the 
event location 31c only when the event type 31b is ^^2" (car 
□ parking), "'3" (car departure) or "5" (person entry) . The event 

.n 10 detection section 2 writes data having meaning (that is, "null") 
=5 as the event location 31c when the event type 31b is '^1" (car 

-■3 entry) and "4" (car leaving) . "null" is written as the event 

L type 31b because it is hard to decide whether the event is 

detected in any of the regions 33a to 33c. 
li^ 15 An event object 31d is a serial number provided to the 

object which performs the event. In this example, the event 
object 31d is the car. The car is identified from the number 
on the number plate of the car. The car number is identified 
by the event detection section 2 as follows. The event 
20 detection section 2 tracks a movement region on the picked up 
image when the event is detected, detects a number plate based 
on the image in the tracked movement region, reads the number 
on the detected number plate by image recognition, identifies 
that the car is an event object having the same serial niomber 
25 if there is the same serial number as the read number, and gives 
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a novel serial number if the number on the number plate is new. 

If the technology of image recognition is used for 
identifying the event object, because it is complicated, a lot 
of time is disadvantageously required. To overcome this 
problem, for example , the event object may be identified by 
recognizing the color of the event object. The objects having 
same color are provided with identical event object 31d. The 
objects having different color are provided with a unique event 
object 31d. The car may be recognized in a different manner. 
For example, a transmitter that transmits the ID of each car 
itself may be mounted in the car. In such a case, the event 
detection section 2 may be constructed so as to receive the ID 
transmitted from the car, and recognize the car based on this 
ID of the car. 

An event time 31e is a time at which the event detection 
section 2 detects an event. The event time shown in Fig. 3 is 
'^19990303 11:12:23". This means that the event is generated 
on March 3, 1999, at 11 o'clock, 12 minutes, 23* seconds. The 
camera ID 31f serves to specify the camera 21 in the case in 
which a plurality of image acquisition sections 1, that is, a 
plurality of cameras 21 are used, by which it is possible to 
know a place where the camera 21 picking up an image to be 
acquired is provided. 

The image recording start time 31g is a time when the image 
pick up is started, as shown in Fig. 3, it is 10 seconds earlier 



with respect to the event time 31e. The image recording end 
time 31h is a time when the image pick up is terminated, as shown 
in Fig. 3, it is 30 seconds after the event time 31e. These 
relative times, that is, 10 seconds and 30 seconds are 
previously set as common parameters for all the event. 

The static image 31i indicates the type of a static image 
acquired at the event time 31e. In Fig. 3 the static image 31i 
is "1-JPG". This means that, the static image is recorded by 
JPEG method. Thus, the static image 31i is referred during 
reproduction. The recorded static image is used as a thumb nail 
of the generated event as will be described later. The dynamic 
image 31j indicates the type of a dynamic image recorded in the 
image recording section 8. In Fig. 3 the dynamic image 31j is 
"l.MPEG". This means that, the dynamic image is recorded by 
MPEG method. Thus, the dynamic image 31 j is referred during 
the reproduction. 

The relationship between the event data created by the 
event data creation section 3 and the image to be recorded in 
the image record section 8 will be explained here by referring 
to Fig. 5. A whole image 34 acquired by the image acquisition 
section 1 is sequentially stored in the temporary storage 
section 8a of the image record section 8. When the event data 
corresponding to the event 35 is created by the event data 
creation section 3, the image recording section 8 records a 
static image at the event time of the event 35 and an event image 



3 6 obtained by extracting an image from the temporary storage 
section 8a from the image recording start time to the image 
recording end time. Thus, only the event image 36, shown with 
blacked out sections, which is only a portion of the whole image 
5 34 is recorded in the image record section 8. When a specific 
period of time elapses, a portion of the whole image 34 other 
than the blacked out portion is erased. 

As shown in Fig. 5, each event 35 contains the event data 
;3 indicative of event ID, event type, event location and event 

.n 10 object from the top in the drawing. One event 35 corresponds 
■M to one event image 36. Accordingly, the retrieval section 6 

^0 can retrieve an image by before and after the time at which each 

;^ event 35 is generated by searching the event data 7a. The 

:I| retrieved image can then be output to and displayed on the 

^ 15 display section 10. 

5 

The macro event creation section 4 structures events 
related to each other from a plurality of event data created 
by the event data creation section 3 based on the macro event 
definition contents 5a defined by the macro event definition 

20 section 5, thereby creating macro event data 7b. The macro 
event definition contents 5a are models of the structure of the 
event and are conditions for creating the macro event data . Fig . 
6 shows an example of the macro event definition contents 5a. 
This macro event definition contents 5a have items such as a 

25 structurizing key 41a, a condition item name 41b and order 
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values 41c to 41g- 

The structurizing key 41a is an item which should 
common to each event when the event is to be structurized , / In 
Fig. 6, the structurizing key 41a is ''event object" 3lA that 
5 corresponds to the event data 7a. This means that,/4:he event 
related to a car having the same number (thatxis, the number 
on the number plate) is to be structurized,.-^ The condition item 
name 41b defines an item to be,/an order condition for 
:3 structurizing the common event^def ined by the structurizing key 

10 41a. In Fig. 6, the condition item name 41b is "event type" 
31b that correspcmd^to the event data 7a. Each of the order 
values 41c tp^^g represent a order value and these are preset . 
L The mi;2^ event is defined by the event type having the order 

i^Jj s^^in the order values 41c to 41g, 

:--=^ 15 In Fig. 6, four order values 41c to 41f are defined, and 

a 

values "1" to "4" of the event type 31b are sequentially defined. 
Assuming that an event previously corresponding to the event 
type 31b is basically generated in order of the values of "1" 
to "4", the value of the event type 31b is set. Therefore, 

20 well-ordered values 41c to 41f are sequentially defined. 
Depending on the order of the generation of the event, these 
order values 41c to 41g are optionally defined. For example, 
the event type of the "car entry" is currently set to "1" and 
the event type of the "car parking" is currently set to "2". 

25 In the case in which the event type of the "car entry" is set 
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to ^^2" and that of the "car parking" is set to "1", the order 
values 41c, 41d, 41e, 41f will be defined as "2", "1", "3", 'M" 
respectively , 

Event processing and a macro event processing 
5 (event/macro event processing) which are carried out by the 
control section C will be described here with reference to the 
flowchart shown in Fig. 7. As shown in Fig. 1, when an event 
is detected by the event detection section 2 (Step Sll) , the 
i!3 control section C carries out the event processing of Steps S21 

=3. 

Ln 10 to S23 and the macro event processing of Steps S31 to S39 in 
parallel and repeats this processing each time the event is 
detected. 

In the event processing, the event data creation section 
3 generates event data based on information about the event 
15 detected by the event detection section 2 (Step S21) . Further, 
the event data creation section 3 records an event image 
corresponding to the event data in the image recording section 
8 (Step S22) . Thereafter, the event data generated at the Step 
S21 is registered as the event data 7a in the management data 
20 section 7 (Step S23) . The process then returns to the Step Sll. 

In the macro event processing, it is decided whether or 
not the detected event has the contents defined as the macro 
event definition contents 5a (Step S31) . More specifically, 
the decision whether or not the detected event has the macro 
25 event definition contents 5a implies the decision whether or 
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not the event object of the structurizing key 41a is identical. 
If the detected event does not have the macro event definition 
contents 5a (Step S31^ NO) , the process returns to the step Sll 
where the detection of a next event is waited. If the detected 
event has the macro event definition contents 5a (Step S31, YES) , 
it is decided whether or not the detected event type has the 
same value as the current order value of the event type to be 
the condition item name 41b (Step S32) . If the event type does 
not have a value same as the current order value (Step S32, NO) , 
it indicates that this particular event is not related to the 
macro event at all. Therefore, the process returns to the Step 
Sll where the detection of a next event is waited. If the event 
type has a value same as the current order value (Step S32, YES) , 
it is decided whether or not the current order value indicates 
first in the order (Step S33) . 

If the current order value indicates first order (Step 
S33, YES) , for example, when the current order has a value shown 
in column of the order value 41c in Fig. 6, the recording of 
a macro event image is started at 10 seconds before the event 
time of the detected event (Step S34) and a continuous macro 
event image is recorded in the image recording section 8 . Then, 
the macro event data at the current point is modified (Step S35) 
and the current order is further updated (Step S36) . Then, the 
process returns to the Step Sll where the detection of a next 
event is waited. 



On the other hand, if the current order value does not 
indicate the first order (Step S33, NO), it is further decided 
whether or not the current order value indicates the last order 
(Step S37) . If the current order value indicates the last order 
5 (Step S37, YES) , for example, when the current order has a value 
shown in column of the order value 41e in Fig. 6, the recording 
of the macro event image which is being recorded is terminated 
after 30 seconds since the detected event time (Step S38) and 
the modified macro event data are registered as the macro event 

10 data 7b in the management data section 7 (Step S39) . Then, the 
process returns to the Step Sll where the detection of a next 
event is waited. If the current order value does not indicate 
the last order (StepS37, NO), the macro event data at the current 
point is modified (Step S35) and the order is further updated 

15 (Step S36) . The process returns to the Step Sll where the 
detection of a next event is waited. In this manner, the event 
data and the event image are recorded and the macro event data 
and the macro event image are recorded. 

The specific relationship between the event data and the 

20 macro event data will be explained here with reference to Fig. 
8 to Fig. 10. Fig. 8 shows an example of the event data including 
an event to be the object of the macro event data. Fig. 9 shows 
an example of the macro event data created corresponding to the 
event data shown in Fig. 8. This macro event data is created 

25 based on the macro event definition contents 5a shown in Fig. 
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6. The macro event data 7b shown in Fig. 9 is created based 
on event data having an event object of "11" and the order value 
of the event type of "1" - "2" - "3" - "4". In other words, 
the macro event data 7b is created from the event data having 
an event ID of '"3", "8", "20" and "22" in the event data 7a shown 
in Fig. 8. 

In Fig. 9, a macro event ID is a serial number provided 
to each of the created macro event. The macro event ID provided 
to the macro event shown in Fig. 9 is "Gl" . A camera ID indicates 
the ID of a camera which is related to the detection of an event 
constituting the macro event and implies a camera for picking 
up the macro event image. An event object indicates an object 
of the event constituting the macro event. According to the 
macro event definition contents 5a shown in Fig. 6, the 
structurizing key 41a is the "event object". Therefore, the 
event constituting the macro event has all the same event 
objects . 

Start time indicates a time at which recording of the 
image of a first event constituting the macro event is started. 
This start time is identical to an image recording start time 
in the event data having an event ID of "3". End time indicates 
a time at which recording of the image of a last image 
constituting the macro event is ended. The end time is 
identical to the image recording end time in the event data 
having an event ID of "22". The item of a dynamic image shows 
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the name given to the macro event image. Structure event IDs 
IDl to IDS indicate the event ID constituting the macro event 
and the number of the structure event IDs can be set as desired. 

Fig. 10 shows the relationship between the event and the 
macro event. The images of events 42a to 42d and 43a to 43d 
which are obtained before and after the event generation time 
for the event corresponding thereto are recorded in the image 
recording section 8 . 

The events 42a to 42d have the same event object '"11" and 
the event type has order values of "1" to "4''. Accordingly, 
the events 42a to 42d constitute the macro event, and a macro 
event image 44 is recorded in the image recording section 8. 
The events 43a to 43d have the same event object of ^^22" and 
the event type has the event values of "1" to "4". Accordingly, 
the events 43a to 43d constitute the macro event and a macro 
event image 45 is recorded in the image recording section 8. 
Consequently, an image in a portion which is originally lost 
is advantageously monitored, recorded and managed efficiently. 

It is also possible to obtain one macro event image from 
a plurality of images acquired by a plurality of image 
acquisition sections 1. When a plurality of image acquisition 
sections 1 are used the amount of information to be acquired 
is increased, however, there is an advantage that monitoring 
can be carried out reliably. Fig. 11 shows an example of the 
case in which three cameras, that is three image acquisition 



sections 1, are used. In Fig. 11, the three cameras have camera 
IDs of ''la", ''lb", and "Ic", respectively. These three cameras 
pick up a images concurrently. 

The camera having the camera ID of "la" detects the event 
5 42a shown in Fig. 10 and stores an event image 36a corresponding 
to the event 42a in the image recording section 8. The camera 
having the camera ID of "lb" detects the events 42b and 42c shown 
in Fig. 10, and stores event images 36b and 36c corresponding 
:3 to the events 42b and 42c in the image recording section 8, 

.n 10 respectively. The camera having the camera ID of "Ic" detects 

'"i 

|3 the event 42d shown in Fig. 10 and store an event image 3 6d 

corresponding to the event 42d in the image recording section 

k 

These events 42a to 42d have only different camera IDs 

:n 

^ 15 and are the same as the events 42a to 42d shown in Fig. 10 in 
other respects. Accordingly, a macro event 4 6 can be 
structurized based on the events 42a to 42d provided across the 
three cameras, and a macro event image corresponding to a macro 
event image 4 4 shown in Fig. 10 can be obtained. For an image 

20 interpolating the event images 3 6a to 3 6d, the next generated 
event is sometimes present on the images picked up by the 
different cameras. Therefore, the image of the camera picking 
up an image having the generated event is recorded as an image 
obtained after the generation of the event. 

25 The event image and the macro event image corresponding 
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to the event data 7a and the macro event data 7b which are thus 
created are recorded in the image recording section 8, and are 
managed by the event data 7a and the macro event data 7b. When 
the information related to an event such as an event type, an 
5 event location, an event object, an event time and the like are 
input from the operation section 9, the retrieval section 6 
refers to the event data 7a and the macro event data 7b in the 
management data section 7 to cause the display section 10 to 
display and output an event and a macro event which are related 
10 to the input information and to give an instruction for 
selecting a desirable event or macro event from the displayed 
and output event group and macro event group. Thus, the event 
image or macro event image to be selected is displayed and 
output . 

15 Fig. 12 shows an example of a graphical user interface 

to be displayed on the display screen of the display section 
10. This graphical user interface also serves as a part of the 
operation section 9. In Fig. 12, an image acquired by the image 
acquisition section 1 in an initial state is displayed and 

20 output on a main screen 54 on a monitor screen 50 in a real time - 
The time 55 indicates the time at which an image displayed 
and output on the main screen 54 was recorded- Every time event 
data or macro event data are created by the event data creation 
section 3, a thumb nail and an event time for the created event 

25 are added to the uppermost part in an event display column 51- 
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By using a scroll bar 52, the event data or macro event data 
thus created can be displayed on the event display coluinn 51. 

By inputting desirable data to items of "event type", 
"event location", "start" and "end" provided in a region El in 
the monitor screen 50 and pressing down a set button, the result 
of retrieval is displayed in the event display column 51. Fig. 
13 is a diagram showing an example of the setting of the region 
El. 

Fig. 13 shows that, the event type is "car parking", the 
event location is "parking area No. 3", the "start" is "12 
o'clock, 10 minutes, January 1, 1999" and the "end" is set to 
"12 o'clock, 10 minutes, January 19, 1999". When such retrieval 
conditions are set, the retrieval section 6 retrieves event data 
and macro event data which satisfy the set retrieval conditions 
of the event type "2", the event location "3" and the event time 
"1999/01/01 12:10:00 to 1999/01/19 12:10:00", and displays the 
result of the retrieval in the event display column 51, The 
display in the event display colxomn 51 includes the thumbnail 
of the event data and the event time. More specifically, only 
the event data are displayed. A star mark 53 displayed on the 
right side of each event data indicates that there is macro event 
data to which the event data in this column belongs. When a 
star mark 53 is absent, it indicates that there is no macro event 
data corresponding to that particular event data. 

Fig. 14 is a diagram showing the state of the monitor 
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screen obtained when a desirable event or macro event is 
selected from the event display column 51. In Fig. 14, when 
event data 57 in the event display column 51 is selected by using 
a cursor 59, the retrieval section 6 fetches an event image 
corresponding to the event data 57 from the image recording 
section 8 and displays and outputs the event image on the main 
screen 54 in the monitor screen 50 of the display section 10. 
To the lower column of the main screen 54 is connected a GUI 
capable of reproducing, temporarily stopping, rewinding and 
feeding the image displayed on the main screen. 

When a the star mark 58 displayed on the right end of the 
event data 57 is selected by using the cursor 59, macro event 
data including the event data 57 are retrieved and are displayed 
and output on the main screen 54 . The display on the main screen 
54 can switch a real-time image, a selected event image or a 
macro event image by the selection of a switch button 56. 
Although the maim screen 54 is displayed in a fixed region in 
the monitor screen 50, another main screen corresponding to the 
main screen 54 may be displayed in a separate frame. 
Furthermore, a plurality of monitor screens may be displayed 
as a multi-window method. 

As explained above, according to the first embodiment, 
macro event data including a related event and having a time 
width are generated in accordance with the macro event 
definition contents, and a macro event image is recorded in 



accordance with the generated macro event data. Therefore, it 
is possible to rapidly and easily acquire significant image 
information which cannot be grasped by only the event image 
obtained before and after the generation of an event. 

A second embodiment of the present invention will be 
explained. While a macro event definition in which a constant 
number of events according to a predetermined order constitute 
a macro event has been carried out in the first embodiment, it 
is possible to carry out a macro event definition in which the 
number of events constituting a macro event is not always 
constant. That is, if certain conditions are satisfied 
variable number of events which number can exceed the constant 
number are permitted. This case is explained below as the 
second embodiment. 

Fig. 15 is a block diagram showing a complete structure 
of an information collection apparatus according to the second 
embodiment. In Fig. 15, a macro event definition section 15 
has macro event definition contents 15a which permit a variable" 
number of events to define a macro event. The management data 
section 7 has macro event data 17b created according to the macro 
event definition contents 15a. Other elements are the same as 
those in the first embodiment and they have been provided with 
the same reference numerals - 

A macro event including an unspecified event defined by 
the macro event definition section 15 will be described below. 



In the first embodiment, the macro event has been defined in 
such a manner that an event type is always constituted by four 
events having an order of '"car entry" - "car parking" - "car 
departure" - "car leaving". However, in the case in which an 
5 event for the entry of unspecified number of people is generated 
between the "car parking" and the "car departure", for example, 
it is supposed that the event of. "person entry" is related to 
the parking situation of a series of cars, and it is desirable 
that the event of "person entry" should be added as a macro event . 

10 When the event of the unspecified number of "person entry" is 
added to the macro event, the event type is constituted in the 
order of "car entry" - "car parking" - "people entry" - "person 
entry" - - "person entry" - "car departure" "car leaving", 
and the unspecified number of events constitute the macro event . 

15 Fig. 16 is a diagram showing an example of the macro event 

definition contents including such unspecified number of events . 
In Fig. 16, the macro event definition contents 15a correspond 
to the macro event definition contents 5a. The value of the 
event type in an order value 61e is not set to "3" but "3:5". 

20 The value of "3" indicates the event type of "car departure" 
and the value of "5" indicates the event type of "people entry". 
In the case in which the event type does not have the value of 
"3" in the order of the order value 61e, and furthermore, the 
value of the event type is "5", the event having the event type 

25 of "5" is registered as an event constituting the macro event. 



In the case in which the event type has the value of ^'3" 
in the order of the order value 61e, an event having the event 
type of "3" is registered as an event constituting the macro 
event and a processing proceeds in the order of a next order 
value 61f . Accordingly, as long as the event type has the value 
of ""5" in the order of the order value 61e, the processing does 
not proceed to the next order but the value of '^3" is repetitively 
checked. In other words, it is apparent that the value of "5" 
in the order value 61e is a condition value for registering the 
event for the event type having the value of '"5" as the macro 
event and for checking the value of '^3" of the order value 61e 
again without proceeding to the next order. 

An event processing and a macro event processing 
(event/macro event processing) which are to be carried out by 
the control section C will be described here with reference to 
the flowchart shown in Fig. 17, As shown in Fig. 17, when an 
event is detected by the event detection section 2 (Step Sll) , 
the control section C carries out the event processing of Steps 
S21 to S23 and the macro event processing of Steps S41 to 352 
in parallel and repeats this processing each time the event is 
detected . 

In the event processing, the event data creation section 
3 generates event data based on information about the event 
detected by the event detection section 2 (Step S21) . Further, 
the event data creation section 3 records an event image 



corresponding to the event data in the image recording section 
8 (Step S22) in the same manner as in the first embodiment. 
Thereafter, the event data generated at the Step S21 is 
registered as event data 7a in the management data section 7 
5 (Step S23) . The process then returns to the Step Sll. 

In the macro event processing, it is decided whether or 
not the detected event has the contents defined as the macro 
event definition contents 15a (Step S41) . More specifically, 
□ the decision whether or not the detected event has the macro 

Lfi 10 event definition contents 15a implies the decision whether or 
not the event object is identical. If the detected event does 
not have the macro event definition contents 15a (Step S41, NO) , 
:L the process returns to the step Sll where the detection of a 

next event is waited. If the detected event has the macro event 
15 definition contents 15a (Step S41, YES), it is decided whether 
or not the detected event type has the same value as the current 
order value (Step S42) . If the event type does not have a value 
same as the current order value (Step S42, NO) , the event is 
not related to the macro event at all. Therefore, the process 
20 returns to the Step Sll where the detection of a next event is 
waited. If the event type has a value same as the current order 
value (Step S42, YES) , it is decided whether or not the current 
order value indicates first in the order (Step S43) . 

If the current order value indicates first order (Step 
25 S43, YES) , the recording of a macro event image is started at 
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10 seconds before the event time of the detected event (Step 
S44) and a continuous macro event image is recorded in the image 
recording section 8 . Then, the macro event data at the current 
point is modified (Step S45) and the current order is further 
5 updated (Step 846) . Then, the process returns to the Step Sll 
where the detection of a next event is waited. 

On the other hand, if the current order value does not 
indicate the first order (Step S43, NO) , it is further decided 
L3 whether or not the current order value indicates the last order 

;=n 10 (Step S47) . If current order value indicates the last order 
'f? (Step S47, YES) , the recording of the macro event image which 

f2 is being recorded is terminated after 30 seconds since the 

;;=^ detected event time (Step S48) and the modified macro event data 

py are registered as the macro event data 17b of the management 

1==^ 15 data section 7 (Step S49) . Then, the process returns to the 

.:=!, 

1 8 

Step Sll where the detection of a next event is waited. 

If the current order value does not indicate the last 
order (Step S47, NO) , it is decided whether or not a condition 
value is added to the order value in the current order (Step 

20 S50) . If the condition value is not added (Step S50, NO), the 
process proceeds to Step S45 where the macro event data at the 
current point is modified (Step S45) . Furthermore, the order 
is updated (Step S46) . Then, process returns to the Step Sll 
where the detection of a next event is waited. 

25 On the other hand, if the condition value is added (Step 



S50, YES), it is decided whether or not the value of the event 
type of the detected event is equal to the condition value (Step 
S51) . If the value of the event type is not equal to the 
condition value (Step SSI, NO) , the process returns to the Step 
Sll where the detection of a next event is waited. If the value 
of the event type is equal to the condition value (Step S51, 
YES) , the macro event data are modified in order to add the 
detected event as an event constituting a macro event (Step S52) 
and the order is not updated. Then, the process returns to the 
Step Sll where the detection of a next event is waited. 

Fig. 18 is a diagram showing an example of the created 
macro event data 17a. The macro event data 17a shown in Fig. 
18 is created based on the event data 7a shown in Fig. 8. Events 
having an event type of "5" and event IDs of '"12" and '"18" are 
added as the macro event. 

According to the second embodiment, an unspecified number 
of events such as '"person entry" are added as the events 
constituting the macro event. Therefore, it is possible to 
create flexible macro event data and to record a macro event 
image corresponding to the macro event data. 

A third embodiment of the present invention will be 
described below. While an image between the first event and 
the last event which constitute a macro event has been 
continuously recorded as a macro event image in the first and 
second embodiments, a macro event image is summarized and 



recorded and an image recording capacity is reduced if a macro 
event continues for a longer time in this third embodiment. 

Fig. 19 is a block diagram showing a complete structure 
of an image collection apparatus according to the third 
embodiment of the present invention. An image summary section 
11 serves to summarize and record a macro event image 
corresponding to macro event data when recording the same macro 
event image in the image recording section 8 . Other elements 
are the same as those in the first embodiment and they have been 
provided with the same reference numerals - 

Fig. 20 is a diagram showing a summary processing carried 
out by the image summary section 11, indicating the relationship 
between an event corresponding to the generation of the macro 
event image 45 shown in Fig. 10 and a macro event. In Fig. 20, 
a macro event 4 6 is obtained by structurizing four events 42a 
to 42d. The macro event image 45 is recorded in the image 
recording section 8 corresponding to the macro event 46. If 
an imaging time for the macro event image 4 5 exceeds a 
predetermined time TMAX, the image summary section 11 selects, 
as macro event images, event images 3 6a to 3 6d corresponding 
to the events 42a to 42d as shown in Fig. 21, and connects the 
event images 3 6a to 3 6d and records them in the image recording 
section 8. Consequently, the macro event image 45 for a longer 
time can be summarized. 

The event images 36a to 36d are recorded corresponding 
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to the events 42a to 42d, respectively. Therefore, the event 
images 3 6a to 3 6d are not newly recorded but they may be connected 
during reproduction by using the macro event data 7a. In this 
case, the event/macro event processing can be implemented by 
means of the control section C by deleting the start of the record 
of a macro event image (Step S34) and the end of the record of 
the macro event image (Step 338) from the Steps S31 to S39 shown 
in Fig. 1, for example. By summarizing the macro event image 
using the event images 36a to 36d, the recording capacity of 
the image recording section 8 can further be reduced. 

Fig. 22 is a diagram showing an example of a macro event 
image interpolated by a static image periodically acquired 
between the event images 3 6a to 3 6d. Fig. 22 corresponds to 
Fig. 20 and shows the relationship among an event, an event image 
and macro event, and a macro event image. As shown in Fig. 22, 
the image summary section 11 does not simply connect the event 
images 36a to 36d but interpolates the event images 36a to 36d 
by a static image 48 periodically acquired between the events, 
and records a summary image 47 to be a macro event image which 
is connected including the interpolated static image 48. 
According to the summary image 47,. the static image 48 is 
interpolated between the event images 36a to 36d. Therefore, 
a change in the situation between the events 42a to 42d can be 
known with a minimum storage capacity and the whole macro event 
can also be grasped easily. 



While the macro event image has been summarized by using 
the event image in the third embodiment, an event image of an 
event other than the event constituting a macro event may be 
recorded as a macro event image. 

When a plurality of image acquisition sections 1 are 
provided and the event is provided across the image acquisition 
sections 1, it is not easy to select a continuous image between 
the images in the first and second embodiments. In the case 
in which the macro event image is summarized by using the image 
summary section 11, it is sufficient that only the event image 
provided across the image acquisition sections 1 is connected. 
Therefore, the summary is particularly effective in the case 
in which a macro event image is to be recorded by using the image 
acquisition sections 1. 

According to the third embodiment, the macro event image 
for a long time is summarized by the image summary section 11 
and the summarized macro event image is recorded. Therefore, 
the recording capacity of the image recording section 8 can be 
reduced . 

A fourth embodiment of the present invention will be 
described below. In the fourth embodiment, data supporting the 
monitor and analysis of an event or a macro event which is 
acquired in the first to third embodiments are measured and the 
measured data are displayed during the display of the event or 
the macro event . 



Fig. 23 is a block diagram showing a complete structure 
of an image collection apparatus according to the fourth 
embodiment. A data measurement section 2 is specifically 
implemented by a GPS receiver. By the GPS receiver, position 
5 information such as latitude, longitude, direction and time is 
measured and output to the control section C. A measurement 
data storage section 13 receives the data measured by the data 
measurement section 12 through the control section C and stores 
the measured data as measurement data 13a and holds map 

10 information 13b. Other elements are the same as those in the 
first embodiment and they have been provided with the same 
reference numerals . 

When an instruction for the display of an event or a macro 
event is given from the operation section 9, the retrieval 

15 section 6 acquires the measurement data corresponding to the 
event or the macro event to which the instruction for the display 
is given and the control section C causes the display section 
10 to display the event or the macro event and to display the 
measurement data. 

20 Fig. 24 is a diagram showing an example of the measured 

data stored in the measurement data storage section 13 . In Fig . 
24, the measured data are recorded as a table for position 
information corresponding to two cameras (image acquisition 
sections 1) having camera IDs of "1" and "2". Measured data 

25 on a latitude, a longitude and a direction are recorded in each 
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table for each time. The "direction" indicates a clockwise 
angle by setting the north to 0 degree. The event or the 
measured data referred at the time of the display of the event 
are obtained by retrieving the camera ID. The time is collated 
with the time of the event to display the position information 
about each camera. 

Fig, 25 is a diagram showing an example of the display 
of the measured data which are displayed together with an event 
image or a macro event image. In Fig. 25, a monitor screen 50 
and a measured data display screen 70 are provided in one display 
region E. The information about the event or the macro event, 
particularly, the event image or the macro event image is 
displayed and output onto the monitor screen 50. The measured 
data on the event information or the macro event image, that 
is, the position information of the camera are displayed onto 
the measured data display screen 70 synchronously with time 
information about the event image or the macro event image which 
is displayed on the monitor screen. 

Map information 13b collated with the position of the 
camera is fetched and an icon 71 of the camera is superposed 
on the map information 13b for the display. Consequently, the 
measured data are specifically displayed and output. The icon 
71 has the shape of a camera for easy recognition, and the 
direction thereof is displayed corresponding to that of the 
measured data 13a. While the monitor screen 50 and the measured 



data display screen 70 are displayed in one display region 
a plurality of display sections 10 may be provided and the 
monitor screen 50 and the measured data display screen may be 
provided individually . 

According to the fourth embodiment, the data related to 
the event or the macro event are measured and the measured data 
are displayed or graphically displayed synchronously with the 
display of the event or the macro event. Therefore, the 
contents of the event or the macro event can easily be understood . 
In particular, in the case in which the image acquisition 
section 1 such as a camera is moved, it is hard to grasp the 
image to be acquired- Therefore, the present embodiment is 
particularly effective . 

A fifth embodiment of the present invention will be 
described below. While the image collection apparatus has 
monitored a parking lot and the like by using an event image 
or a macro event image in all the first to fourth embodiments, 
an event image or a macro event image is analyzed by using an 
event image in the fifth embodiment. 

Fig. 2 6 is a block diagram showing a complete structure 
of the image collection apparatus according to the fifth 
embodiment of the present invention. An analysis section 14 
carries out a predetermined analysis processing based on an 
event image and a macro event image which are recorded in an 
image recording section 8. Other elements are the same as those 




in the first embodiment and they have been provided with the 
same reference numerals. 

The analysis section 14 analyzes the behavior of a car. 
As shown in Fig. 21 , therefore, a camera 81 acting as an image 
5 acquisition section 1 is provided on the road side of a circuit 
course 84, and a sensor 82 acting as an event detection section 
2 is similarly installed in the vicinity of the camera 81. The 
sensor 82 reads the speed and acceleration speed (deceleration 
speed) of a car 83 running on the circuit course 84 and a number 

10 drawn on the body of the running car 83. 

An event type constituting event data includes four, that 
is, ''car entry", "deceleration start", "deceleration end" and 
"car leaving". Accordingly, it is possible to create a macro 
event by structurizing a plurality of events having the same 

15 number and corresponding to the event type. 

In the circuit, a large number of cars run around the same 
course many times. Also in the corner in which the camera 81 
is installed, therefore, a large number of cars pass many times 
while decelerating. Thus, a macro event image related to a 

20 series of movements is recorded. Accordingly, a macro event 
image related to a series of events including "car entry"- 
"deceleration start" - "deceleration end" - "car leaving" on 
this side of the corner of a specific car can easily be retrieved 
from the image picked up by the camera 81 where a plurality of 

25 cars are viewed, and can be displayed and output. 
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The analysis section 14 analyzes the behavior of the car 
based on the information about the macro event thus obtained. 
In that case, since the information is recorded as the macro 
event, the analysis processing can be carried out rapidly. 

According to the fifth embodiment, the analysis section 
14 can carry out an analysis processing by using the information 
about the macro event. Therefore, the analysis processing can 
easily be carried out. The first to fifth embodiments can be 
combined properly - 

According to one aspect of the present invention, as 
described above, the image acquisition unit acquires an image, 
the event detection unit detects an event, and the event data 
creation unit creates event data indicative of at least a 
generation time and contents of the event detected by the event 
detection unit. The macro event definition unit which defines 
a macro event including a related event group and having a time 
width by using the generation time and contents of the event, 
the macro event data creation unit creates macro event data by 
structurizing a plurality of event data based on definition 
contents of the macro event, and the image recording unit 
records an' image of a macro event corresponding to the macro 
event data. The retrieval unit retrieves a desirable macro 
event based on the macro event data, and the display unit 
acquires at least an image of the macro event retrieved by the 
retrieval unit from the image recording unit and displays the 



image. Consequently, it is possible to rapidly and easily 
obtain image information which cannot be grasped by only the 
event image before and after the event generation and is 
significant and inclusive. 
5 Further, the temporary storage unit temporarily stores 

the image acquired by the image acquisition unit, the image 
recording unit extracting and recording an image related to an 
event detected by the event detection unit from the image 
□ temporarily stored by the temporary storage unit. 

■ rj 

=n 10 Consequently, the recording capacity of the image recording 
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'3 unit can be reduced so that monitoring for a longer time can 

be carried out. In addition, it is possible to implement an 
image collection apparatus which is light and small-sized. 

Further, the macro event definition unit carries out 

^ 15 definition including repetition of a predetermined event in a 

a 

series of events constituting the macro event and the macro 
event data creation unit creates macro event data in accordance 
with the definition contents. Consequently, it is possible to 
easily obtain a flexible image of the macro event. 

20 Further, the image summary unit selects an image in a 

predetermined portion of the image of the macro event and 
summarizes the image of the macro event, the image recording 
unit recording the image of the macro event summarized by the 
image summary unit. Consequently, the recording capacity of 

25 the image recording unit can be reduced so that monitoring for 



a longer time can be carried out. In addition, it is possible 
to implement an image collection apparatus which is light and 
small-sized . 

Further, the . image summary unit selects an image in a 
predetermined portion of the image of the macro event, 
periodically acquires a static image from a non-selected image 
of the macro event and connects the selected image with the 
static image on a time basis, thereby summarizing the image of 
the macro event, the image recording unit recording the image 
of the macro event summarized by the image summary unit, and 
particularly, intermittently interpolating the images 
selected by the static image. Consequently, the non-selected 
image portion with the siimmary can be complemented ef ficiently. 
Furthermore, the recording capacity of the image recording unit 
can be reduced so that monitoring for a longer time can be carried 
out. In addition, it is possible to implement an image 
collection apparatus which is light and small-sized. 

Further, the measurement unit measures data related to 
the image, and the measured data storage unit stores the 
measured data acquired by the measurement unit in relation to 
the macro event, the display unit displaying the image of the 
macro event and displaying the measured data related to the 
macro event, thereby supporting to understand the contents of 
the image of the macro event. Consequently, it is easy to 
understand the contents of the event image or the macro event 



image. In particular, in the case in which the image 
acquisition unit is moved, the position information is set to 
the measurement data so that it is possible to easily understand 
the contents of the event image or the macro event image. 

Further, the image recording unit selects one of the image 
acquisition units corresponding to the image of the macro event 
from a plurality of images input from a plurality of image 
acquisition units and records, as the image of the macro event, 
the image input from the selected image acquisition unit. 
Consequently, also in the case in which the image is to be 
acquired by a plurality of image acquisition units, it is easy 
to obtain the macro event image . 

Further, the analysis unit carries out an analysis 
processing such as the analysis of the behavior of a car based 
on at least the image of the macro event. Therefore, in the 
case in which the predetermined analysis processing is to be 
carried out based on the collected image, it is possible to carry 
out the analysis processing which is more inclusive and 
efficient . 

According to another aspect of the present invention, an 
image is acquired and an event is detected at the acquiring step, 
event data indicative of at least a generation time and contents 
of the event detected at the acquiring step are created at the 
event data creating step, macro event data are created by 
structurizing a plurality of event data generated by the event 
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data generation unit based on macro event definition contents 
defining a macro event including a related event group and 
having a time width at the macro event data creating step, an 
image of a macro event corresponding to the macro event data 
is recorded at the image recording step, a desirable macro event 
is retrieved based on the macro event data at the retrieving 
step, and at least the image of the macro event retrieved at 
the retrieving step is acquired and displayed at the displaying 
step- Consequently, it is possible to rapidly and easily obtain 
image information which cannot be grasped by only the event 
image before and after the event generation and is significant 
and inclusive. 

Further, the image acquired at the acquiring step is 
temporarily stored at the temporary storing step, the image 
recording step extracting and recording an image related to an 
event detected at the acquiring step from the image temporarily 
stored at the temporary storing step. Consequently, the 
recording capacity of the image recording unit can be reduced 
so that monitoring for a longer time can be carried out. In 
addition, it is possible to implement an image collection 
apparatus which is light and small-sized. 

Further, the macro event creating step creates macro 
event data based on the macro event definition contents defined 
including repetition of a predetermined event in a series of 
events constituting the macro event. Consequently, it is 



possible to easily obtain a flexible image of the macro event. 

Further, an image in a predetermined portion of the image 
of the macro event is selected and the image of the macro event 
is summarized at the image summarizing step, the image recording 
step recording the image of the macro event summarized at the 
image summarizing step. Consequently, the recording capacity 
of the image recording unit can be reduced so that monitoring 
for a longer time can be carried out. 

Further, at the image summarizing step, an image in a 
predetermined portion of the image of the macro event is 
selected and a static image is periodically acquired from a 
non-selected image of the macro event and the selected image 
is connected with the static image on a time basis, thereby 
summarizing the image of the macro event, the image recording 
step recording the image of the macro event summarized at the 
image summarizing step. Consequently, the non-selected image 
portion with the summary can be complemented efficiently. 
Furthermore, the recording capacity of the image recording unit 
can be reduced so that monitoring for a longer time can be carried 
out . 

Further, data related to the image is measured at the 
measuring step, and the measured data acquired at the measuring 
step is stored in relation to the macro event at the data storing 
step, the displaying step displaying the image of the macro 
event and displaying the measured data related to the macro 



event. Consequently, it is easy to understand the contents of 
the event image or the macro event image. In particular, in 
the case in which the image acquisition unit is to be acquired 
by movement, the position information is set to the measurement 
data so that it is easy to understand the contents of the event 
image or the macro event image . 

Further, the image recording step selects one of the 
images corresponding to the image of the macro event from a 
plurality of input images and records the selected image as the 
image of the macro event. Consequently, also in the case in 
which a plurality of images are to be acquired at the same time, 
it is easy to obtain the macro event image. 

Further, at the analyzing step, an analysis processing 
such as the analysis of the behavior of a car is carried out 
based on at least the image of the macro event. Therefore, in 
the case in which the predetermined analysis processing is to 
be carried out based on the collected image, it is possible to 
carry out the analysis processing which is more inclusive and 
efficient . 

Although the invention has been described with respect 
to a specific embodiment for a complete and clear disclosure, 
the appended claims are not to be thus limited but are to be 
construed as embodying all modifications and alternative 
constructions that may occur to one skilled in the art which 
fairly fall within the basic teaching herein set forth. 



